Information-Greedy Global Optimization

Optimization is about inferring the location of the optimum of a function. An information-optimal optimizer should thus aim to collapse its belief about the location of the optimum towards a point-distribution, as fast as possible. But the state of the art rarely addresses this inference problem. Instead, it usually relies on some heuristic predicting function optima, then evaluates at the maximum of the heuristic. The reason there are no truly probabilistic optimizers yet is that they are intractable in several ways. In this paper, we present tractable approximations for each of these issues, and arrive at a flexible global optimizer for functions under Gaussian process priors, which performs well in comparison to a state of the art Gaussian process optimizer.